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7J  Background 

Challenge: 

Ability  to  clearly  and  concisely  capture  the  status  of  the 
software  being  developed  for  the  F-35  Lightning  II  Joint 
Strike  Fighter  (JSF)  for  Program  Management  and  external- 
to-the-Program  consumption. 

•  There  are  numerous  teams  responsible  for  developing  and  integrating 
software 

•  There  are  differences  in  the  software  development  processes  being 
executed  across  the  teams 

•  There  is  a  wide  variety  of  software  metrics  being  collected,  analyzed, 
and  reported 

•  The  ability  to  successfully  capture  this  status  lies  with  the  melding  of 

the  data  from  the  numerous  teams ,  processes ,  and  metrics ,  along 

with  the  different  domains  of  software  ( airborne  and  ground  systems ) 
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The  General’s  Four  Questions 

•  Getting  It  Done? 

>  AS  Software  Progress  Summary  and  Block  X  Software  Summaries 

-  Software  Effort  by  Block/Product  Domain 

-  Accomplishments  (last  3  months)  and  Current  Challenges 

-  Block/Product  Percent  Complete  (plan  vs.  actual) 

>  AS  Software  and  Block  X  Progress  to  Plan  Charts 

-  Track  Percent  Complete  by  CY  Quarter  (plan  vs.  actual) 

•  Getting  It  Done  Efficiently? 

>  Productivity  metric  for  selected  high  risk  products  based  on  Effective 
Source  Lines  of  Code  (ESLOC)/hour 

•  Getting  It  Done  Right? 

>  Defect  prediction  metrics  based  on  product-level  defect  profiles 

•  Getting  It  Working  (Fixing  It)? 

>  Utilizing  existing  SPAR  burn  down/status  metric  charts 
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Technical  Issues  for  Metrics 


LIGHTNING  II 


•  Significant  Diversity  in  Development 

-  80+  Teams  -  Prime  Team  and  Suppliers 

-  No  common  source  of  data  for  all  developers 

-  Different  methodologies,  tools,  base  data 

•  Disparate  schedules 

-  Air  Vehicle  schedules,  blocks  different  than  ground-based 
systems 

•  Expense  of  low  level  collection 

-  Discrete  data  collection  required 

•  Different  target  audience 

-  Generals,  Admirals,  Vice  Presidents 

-  Need  to  focus  on  “Big  Picture” 

-  Existing  metrics  focused  on  detail  at  the  Product  Team  levels 
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Metrics  Approach 

•  Normalize  around  ESLOC 

-  Effective  Source  Lines  of  Code  proportional  to  development  effort 

-  Used  with  “%  Complete ”,  produces  mathematically  valid  results 

-  Valid  weighting  factor  for  producing  composite  results 

•  Collect  data  from  Product  Domain  Points  of  Contact 

-  Common  data  collection  form  with  Product  Domain  summaries 

-  Life  Cycle  includes  Requirements  through  Air  System  Integration 

-  Map  everything  to  Air  Vehicle  Blocks  for  reporting 

•  Exploit  existing  summary  reports 

•  Publish  Quarterly  Reports 

-  Driven  by  Executive  briefings  (Quarterly,  Semi-Annual) 

-  Focus  on  big  picture  -  hyperlink  to  details  (optional  usage) 

-  Provide  significant  detail  in  Notes  pages 

•  Examples  of  key  charts  with  notional  data  follow 
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Progress  Chart 


Summary 


LIGHTNING  II 


Block  A  Block  B  Block  C  Block  D  Block  E 


Actual  SLOC  produced  as  of  12/31/06 


6.89M 
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Progress  Chart  -  Product  Domain 


LIGHTNING 
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-  MS  variance  further  reduced  to  1% 

-  OMS  released  increment  3  for 
integration  and  test 

Current  Challenges 

-  Resource  contention  between  blocks  £ 
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Block-Level  Progress  Plan 


Block  X  Progress  to  Plan 
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Sample  Productivity  Chart 


LIGHTNING  II 


Development  Productivity 

(Requirements,  Development,  CSCI  Test,  Rework) 
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Sample  Defect  Prediction 

Cumulative  Software  Defects 
Data  as  of  End  of  Feb-07 


LIGHTNING  II 


l  IFF - Model  - Lower  Limit - Upper  Limit  X  Actual 
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Sample  Defect  Resolution  Charts 


LIGHTNING  I 

X3/ 


Report  Date:  03/26/07 


Air  Systems  SPARs  Identified  in  Block 


Block  Level  SPAR  Charts  to 
assess  open/close  rates 


All  Severity: 

Opened: 

Closed: 

16  week  Open  Rate: 

16  week  Closure  Rate: 

Total  Remaining  Open  SPAR: 
Total  SPAR: 


(Sewirity  1  &  2's  Plus  50%  3's) 


/  /  /  /  /  /  /  /  /  /  /  /  J  / 
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Severity  1  l  l  Severity  2  ■  Severity  3  Severity  4  CH 


Three  levels  of 
Defect  Resolution 
metrics  are  used, 
depending  on  time- 
to-release 


Release-Level  SPAR  Charts 
assess  capacity  to  close 


All  Severities  -  Status  View  as  of  25-Mar-07 


— AV  Open  SPARs  severity  1  &  2's  plus  50%  3‘s 
— AV  Capacity  History  (16-week  Close  Rate) 

♦  AV  Max  Closed  History  (4-week  Close  Rate) 


-  MS  Open  SPARs  severity  1  &  2's  plus  50%  3's 
—VS  Capacity  History  (16-week  Close  Rate) 

♦  VS  Max  Closed  History  (4-week  Close  Rate) 


Release-Level  SPAR  Charts  to 
track  burndown  plans 


Report  Date:  25-Mar- 
2007 

All  Severities: 

Opened: 

16  week  Open  Rate: 

16  week  Closure  Rate: 

Total  Open: 

Severity  1 .2.  &  50%  3’s: 

Opened: 

16  week  Open  Rate: 

16  week  Closure  Rate: 

Total  Open  Sev  1.2.50% 


□  Pending  Verify  Concur 

□  Verification 

■  Pending  Release 

□  Implementation 

□  Pending  Analysis  Concur 

■  Analysis 

■  Identification 
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Satisfied  Customers 

•  The  Air  System  Software  Roll-Up  Metrics  Package  is  being 
developed  per  Program  Management  (Government  and 
Prime  Contractor)  direction 

•  The  Package  successfully  produced  for  last  five  quarters. 

•  The  Package  has  matured  over  time  and  the  metrics 
contents  will  continue  to  evolve  as  the  Program  evolves. 

•  Each  Package  has  been  successfully  briefed  to  Program 
Management  (Government  and  Prime  Contractor). 

•  Contents  of  each  Package  has  been  used  as  input  to 
executive  briefs 

-  Chief  Executive  Officer  (CEO)  Conferences 

-  Configuration  Steering  Board/Software  Acquisition  Executive 
meetings 

-  Joint  Executive  Steering  Board  meetings 
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Lessons  Learned 

•  Generals,  Admirals,  and  Vice  Presidents  have  different 
information  needs 

-  Shape  a  metrics  package  that  will  addresses  their  needs 

-  Be  flexible  -  make  changes  in  response  to  Executives  desires 

•  Use  a  mechanized  and  consistent  method  to  generate 
metrics  to  eliminate  man-in-the-loop  effects  on  the  data. 

•  Identify  data  providers  from  each  development  area  and  train 
them  on  the  data  collection/metrics  generation  process. 

-  Data  providers  must  take  ownership  of  the  data  that  they  provide 
and  provide  backup  data,  when  requested. 

•  Plans  Change,  Estimates  Change  -  be  prepared  to  explain 

•  Avoid  mixing  data  with  different  frequencies. 

•  Summing  up  productivity  masks  information. 

•  Extrapolation  has  risks. 
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Next  Topic 


LIGHTNING  II 


Monday,  June  18,  2007,  track  2 

•  2:40-3:25  -JSF  Software  Program 

-  Overview  and  Status  (Branyan/Willis) 

-  Tracking  Program  Wide  Software  Progress  (Evers/Willis) 

•  3:35-4:20  -  System/Software  Design 

-  Implementing  F-35  System  Architecture  using  UML  (Clauss) 

-  Deploying  C++  For  Use  In  International  Safety-Critical  Applications  (Carroll) 

•  4:20  -  4:35  -  Break 

•  4:35  -  5:20  -  JSF  Software  Safety  Process 

-  Deploying  Safety  Critical  Standards  Internationally  (Eccles) 

-  Providing  Developmental  Assurance  (Bridges) 

•  5:30  -  6:1 5  -  Software  Quality  Improvements  /JSF  Software  Sustainment 

-  Focused  Software  Quality  Improvements  (Robb) 

-  F-35  Software  Life-cycle  Planning:  Performance-Based  Software  Sustainment 
(Novak) 
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Acronyms 


LIGHTNING  II 


Acronym 

Definition 

MS 

Mission  Systems 

OMS 

Off-Board  Mission  Support 

PHM 

Prognostic  Health  Management 

SLOC 

Source  Lines  of  Code 

SPAR 

System  Product  Anomaly  Report 

VS 

Vehicle  Systems 

Acronym 

Definition 

AA-1 

First  Flight  Test  aircraft  nomenclature 

AF-1 

First  Air  Force  variant  aircraft  nomenclature 

ALIS 

Autonomic  Logistics  Information  System 

AVI 

Air  Vehicle  Integration 

CEO 

Chief  Executive  Officer 

CF-1 

First  Carrier  (Navy)  variant  aircraft  nomenclature 

CSCI 

Computer  Software  Configuration  item 

CY 

Contract  Year 

ESLOC 

Effective  Source  Lines  of  Code 

FF 

First  Flight 

FTU 

Flight  Test  Update 

HMD 

Helmet  Mounted  Display 

JPO 

JSF  Program  Office 

JSF 

Joint  Strike  Fighter 
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